Solving Sequential Decision-making Problems under Virtual Reality Simulation System

A large class of problems of sequential decision-making can be modeled as Markov or Semi-Markov Decision Problems, which can be solved by classical methods of dynamic programming. However, the computational complexity of the classical MDP algorithms, such as value iteration and policy iteration, is prohibitive and will grow intractably with the size of problems. Furthermore, they require for each action the one step transition probability and reward matrices, which is often unrealistic to obtain for large and complex systems. Here, we provide the decision-maker a sequential decision-making enviroment by establishing a virtual reality simulation system, where the uncertainty property of system can also be shown. In order to obtain the optimal or near optimal policy of sequential decision problem, simulation optimization algorithms as infinitesimal perturbation analysis are applied to complex queuing systems. We present a detailed study of this method on the sequential decision-making problem in Boeing-737 assembling process.